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Appendix A. About the reading assessments 


This appendix provides information on the reading assessments administered by District U-46 (Elgin Area Schools) 
and District 186 (Springfield Public Schools). 


The two districts in the study assessed students’ reading achievement multiple times in kindergarten and grade 1. 
To track reading achievement, District U-46 uses the Fountas and Pinnell Benchmark Assessment System (Fountas 
& Pinnell, 2007), and District 186 uses the Measures of Academic Progress for Primary Grades (Northwest 
Evaluation Association, 2009). District U-46 data include the percentage of students who are meeting reading 
proficiency benchmarks by spring of grade 1 but do not provide comparisons with national norms. In contrast, the 
District 186 continuous score captures more nuance about reading achievement because it uses a standardized 
score at each time point, with trends in reading achievement in District 186 that can be compared with students 
nationally. The two assessments are described in detail below. 


District U-46 assesses reading achievement in kindergarten and grade 1 using the Fountas and 
Pinnell Benchmark Assessment System 

The Fountas and Pinnell Benchmark Assessment System (Fountas & Pinnell, 2007), a formative reading assessment 
administered by teachers, provides information on the following early reading proficiencies: decoding, fluency, 
vocabulary, and comprehension. This assessment contains items that identify how many letters students 
recognize by name or sound and what students know about letter—sound identification. The student reads aloud 
from the Fountas and Pinnell book while the teacher observes and notes the student’s responses. The teacher 
then conducts a comprehension conversation with the student (Fountas & Pinnell Literacy, n.d.). Teachers give 
each student an achievement rating from A to Z (Fountas & Pinnell, 2014). 


Teachers determine eligibility for the Fountas and Pinnell reading assessment based on a number of factors. 
Before students can take the Fountas and Pinnell reading assessment, they must pass a district-created literacy 
screener in kindergarten. This screener contains items that identify how many letters a student recognizes by 
name or sound and what the student knows about letter—sound identification. Some students pass the literacy 
screener in fall of kindergarten and may take the Fountas and Pinnell Benchmark Assessment as early as winter 
of kindergarten, whereas other students take the assessment later in the year. General education students in 
kindergarten who receive instruction in English needed a letter—sound score of at least 17 in fall of kindergarten 
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to take the Fountas and Pinnell Benchmark Assessment in winter. Dual language education students receiving 
instruction in both English and Spanish needed a letter—sound score of at least 16 in fall of kindergarten to take 
the assessment in winter. Kindergarten students who did not score at or above these levels on the letter-sound 
assessment in fall took the assessment again in winter. For kindergarten students a passing score on the letter— 
sound screener in winter was 21 if they were in the general English program and 20 if they were in the Spanish- 
dominant one-way or two-way dual language program. All kindergarten students in the English-dominant dual 
language two-way program had to take the letter-sound screener in Spanish in fall and winter of kindergarten, 
regardless of their fall score. All students were eligible to take the Fountas and Pinnell Text Benchmark Assessment 
in winter and spring of grade 1. Students were given the assessment in fall of grade 1 only if they did not have a 
score in spring of kindergarten or if their Fountas and Pinnell Benchmark Assessment score was at level B or below. 


Teachers administered the Fountas and Pinnell Benchmark Assessment in winter and spring of kindergarten, and 
again in winter and spring of grade 1, to all kindergarten and grade 1 students. In fall of grade 1 teachers 
administered the assessment only to grade 1 students who scored below the threshold for proficiency in spring 
of kindergarten. Students took the Fountas and Pinnell Benchmark Assessment System at approximately the same 
time in each administration period. Testing in winter and spring sessions took place in early February each year 
(January 31—February 14 of 2017 and 2018) and mid-May each year (May 12—26, 2017, and May 10-24, 2018). 


Teachers scored student performance on the reading assessment using the Fountas and Pinnell Text Level 
Gradient, assigning a letter rating from A to Z representing the level of reading proficiency for a student at a given 
time. Students are considered proficient if they meet the time-specific milestone on any version of the assessment 
(English or Spanish): level C in winter of kindergarten, level D in spring of kindergarten, level F in winter of grade 
1, and level J in spring of grade 1. The measure has a Cronbach’s alpha of .93 and convergent reliability of r= .93 
to .94 with similar measures (Heinemann, 2008). 


District 186 assesses reading achievement in kindergarten and grade 1 through the Measures of 
Academic Progress for Primary Grades 


The nationally normed Measures of Academic Progress for Primary Grades is adaptive and appropriate for 
measuring growth in students’ reading achievement (Northwest Evaluation Association, 2009). The assessment 
measures three literacy achievement domains: phonological awareness, phonics, and concepts of print. 


Students take the assessment by computer and complete it in 20-40 minutes on average; educators are 
encouraged to give students breaks as needed (Northwest Evaluation Association, 2009). A warm-up assessment 
is available to help students get accustomed to interacting with a computer. Developers limited the screen 
interaction functions in the assessment to moving a mouse across the screen and left-clicking. The warm-up 
application and limited functions help ensure that students without computer experience are able to take the 
assessment (Northwest Evaluation Association, 2009). District 186 administers the assessment three times a 
year—in fall, winter, and spring—from kindergarten through grade 2. 


Reading scores are generated automatically when a student completes the assessment. The scores are reported 
as Rasch units. The testing model underlying the Measures of Academic Progress for Primary Grades is a one- 
parameter item-response theory model, and the Rasch unit scale is a transformation of the Rasch ability estimates. 
Scores range from 100 to 300. The technical manual for the assessment gives detailed information about the 
validity and reliability of the assessment (Northwest Evaluation Association, 2009). The Measures of Academic 
Progress for Primary Grades reading assessment is internally consistent, with Cronbach’s alphas ranging from .67 
to .93. The measure has test-retest correlations ranging from r= .71 to r= .87 and convergent reliability of r= .64 
to .81 with similar measures. 
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Appendix B. Methods 


This appendix contains information about the study sample for District U-46 (Elgin Area Schools) and District 186 
(Springfield Public Schools) and the methods used for the descriptive analyses. 


The study sample for District U-46 included 2,703 students in kindergarten and grade 1 


District U-46 only recently began to digitize the data on students’ performance on the Fountas and Pinnell 
Benchmark Assessment System (Fountas & Pinnell, 2007). As a result, the study team had access to data only on 
the cohort of students who were in kindergarten in 2016/17 and who transitioned to grade 1 in 2017/18. Thus, 
the study timeframe included a maximum of four measurement occasions for the 2016/17 cohort of kindergarten 
students: winter and spring of kindergarten and winter and spring of grade 1. Although students who scored a B 
or below on the Fountas and Pinnell Benchmark Assessment in spring of kindergarten were tested again in fall of 
grade 1, these data were excluded from the analyses because only a subset of students was tested. The total 
sample for the analyses of District U-46 data thus consisted of 2,703 students in kindergarten and grade 1 
(table B1), including 2,396 students in winter of kindergarten, 2,420 students in spring of kindergarten, 2,586 
students in winter of grade 1, and 2,582 students in spring of grade 1. 


Table B1. Demographic characteristics of grade 1 students in the analytic samples for District U-46 and District 
186 in Illinois 


District U-46 (grade 1 in 2017/18) District 186 (spring of grade 1 in 2019) 
1BY=Yaalo):4g-] 0) gl (oman e LU 0) Sample Percent Sample Percent 
Asian students 217 8.03 23 2.48 
Black students 153 6.66 404 43.53 
Hispanic students 1,540 56.97 35 3.77 
White students 698 25.82 332 35.78 
Multiracial students 83 3.07 131 14.12 
Other race/ethnicity students 12 0.44 3 0.03 
Students eligible for the national 
school lunch program 1,725 63.82 678 73.06 
Students not eligible for the national 
school lunch program 978 36.18 150 16.16 
English learner students 1,412 52.24 na na 
Non-English learner students 1,291 47.76 na na 
Students in special education 310 11.47 202 21.77 
Students not in special education 2,393 88.53 626 67.46 
Female students 1,362 50.39 449 48.38 
Male students 1,341 49.61 479 51.62 
Total number of students 2,703 100.00 928 100.00 


na is not applicable. 

Note: The sample for District U-46 represents all grade 1 students enrolled in the 2017/18 school year who took a reading assessment at any time point. 
The sample for District 186 represents only students enrolled in spring of grade 1 in the 2018/19 school year who took a reading assessment at any time 
point. Percentages may not sum to 100 because of rounding. 

Source: Authors’ analysis of assessment and student records data for 2016/17—2017/18 from District U-46 and for 2017/18-2018/19 from District 186. 


Students must meet district-defined criteria to be eligible to take the Fountas and Pinnell Benchmark Assessment 
(see appendix A). District U-46 allows students to demonstrate proficiency on either the English or the Spanish 
version of the assessment. If a student passed at least one version of the assessment, the study team included 
that student’s score in the sample for that time point. (See figure C1 and table C2 in appendix C for supporting 
information.) All grade 1 students were eligible to take the Fountas and Pinnell Benchmark Assessment in winter 
and spring. 


REL 2021-058 B-1 


The study sample for District 186 included 928 students in kindergarten and grade 1 

District 186 began administering the Measures of Academic Progress for Primary Grades to kindergarten students 
in 2017/18. The study time frame allowed for measurement at six time points: fall, winter, and spring of 
kindergarten in 2017/18 and fall, winter, and spring of grade 1 in 2018/19. At the teacher’s discretion, a student 
could take the assessment multiple times in a single testing period. The study team used the score with the largest 
standard error (when available), the criterion used by the district to select a score when a student had multiple 
scores. When multiple assessments had the same standard error, the score of the most recent assessment was 
used in the analysis, following the suggestion of district administrators. 


The total sample size for the analyses of District 186 data consisted of 928 students in kindergarten and grade 1 
(see table B1). The sample included 927 students in fall of kindergarten, 947 students in winter of kindergarten, 
927 students in spring of kindergarten, 930 students in fall of grade 1, 929 students in winter of grade 1, and 928 
students in spring of grade 1. 


To calculate the percentage of students in District U-46 who met the reading proficiency milestone at 
each time point, the study team divided the number of students who met the milestone by the 
number of students with nonmissing assessment scores at each time point 

To address research question 1, the study team calculated the percentage of students in District U-46 who met 
the reading milestone at each time point. The numerator was the number of students who met the spring of grade 
1 reading proficiency milestone (level J) at each time point, and the denominator was the number of students 
with assessment scores at each time point. The study team conducted supplemental analyses using a stable 
sample, which included only students for whom reading achievement scores were available at all four time points 
(see appendix D). 


To address research question 2, the study team used binary indicators of membership in the following student 
demographic groups: Asian, Black, Hispanic, White, multiracial, or other race/ethnicity each relative to another 
racial/ethnic group; eligible for the national school lunch program or not eligible; English learner student or non— 
English learner student; in special education or not in special education; and male or female. The “other 
race/ethnicity” category combined race/ethnicity categories that each represented less than 2 percent of the total 
number of students in the study. The study team calculated the percentages of students from each demographic 
group who met the reading proficiency milestone across successive time periods by dividing the number of 
students in the demographic group who met the spring of grade 1 proficiency milestone (level J) at each time 
point by the number of students in the demographic group with assessment scores at each time point. 


To calculate reading achievement in District 186, the study team computed the average Rasch unit 
score on the assessment at each time point 

To address research question 1, the study team calculated the average Rasch unit score on the Measures of 
Academic Progress for Primary Grades (Northwest Evaluation Association, 2009), standard deviation, interquartile 
range, and range at each time point for students in District 186. The study team conducted supplemental analyses 
using a stable sample, which included only students with reading achievement scores at all six time points (see 
appendix D). 


To make differences between demographic groups more meaningful for practitioners, the study team reported 
differences at the last time point (spring of grade 1) as effect sizes (Hedges’ g) and then translated differences into 
days of instruction. The study team calculated days of instruction by dividing the difference in means by the mean 
effect size for annual achievement in reading across seven nationally normed assessments and then multiplying 
the result by 180 days. For assessment scores from kindergarten, the study team used the mean effect size for 
annual achievement in reading in kindergarten across seven nationally normed assessments, or 1.52 standard 
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deviation units. For assessment scores from grade 1, the study team used the mean effect size for annual 
achievement in reading in grade 1 across seven nationally normed assessments, or .97 standard deviation units. 


To answer research question 2, the study team created binary indicators of membership in the following 
demographic groups: whether a student was Asian, Black, Hispanic, White, multiracial, or other race/ethnicity 
each relative to another racial/ethnic group; eligible for the national school lunch program or not eligible; in 
special education or not in special education; and male or female. The “other race/ethnicity” category combined 
racial/ethnic categories that each represented less than 2 percent of the total number of students in the study. 
The study team then compared the average Rasch unit scores on the Measures of Academic Progress for Primary 
Grades across demographic groups. 
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Appendix C. Supporting analyses 

This appendix includes supporting analyses for the report findings. District U-46 (Elgin Area Schools) has different 
testing procedures for Spanish speaking students and a larger number of English learner students than District 
186 (Springfield Public Schools). Therefore, the appendix includes supporting analyses related to Spanish speaking 
students and English learner students only for District U—46. 


Supporting analyses for District U-46 include detailed information by race/ethnicity on the number of students 
who took the assessment at each time point and the percentage of students who met the spring of grade 1 reading 
proficiency milestone (level J) at each time point. These analyses also include information on how many students 
took the English or Spanish version only and how many students took both versions of the assessment at each 
time point. In addition, the study team calculated the number and percentage of Hispanic students in District U- 
46 who also were English learner students and Hispanic students who were non-English learner students who 
met the spring of grade 1 reading proficiency milestone. Finally, the study team examined the percentage of 
students in District U-46 eligible for the national school lunch program, by race/ethnicity, and the percentage 
eligible for the program who met the spring of grade 1 proficiency milestone at each time point, by race/ethnicity. 


Supporting analyses for District 186 included an examination of average assessment scores of students eligible for 
the national school lunch program, by race/ethnicity. 


The percentage of students in District U-46 who met the spring of grade 1 reading proficiency 
milestone at each time point varied by race/ethnicity 

The main report contains information about the percentage of Asian, Black, and Hispanic students in District U- 
46 who met the spring of grade 1 reading proficiency milestone by the end of grade 1. Additional detail about 
students of other races/ethnicities shows that 61 percent of White students, 56 percent of multiracial students, 
and 36 percent of other race/ethnicity students met the spring of grade 1 reading proficiency milestone by the 
end of grade 1 (table C1). The starkest contrast in reading proficiency at spring of grade 1 was between Asian 
students (73 percent) and other race/ethnicity students (36 percent). 
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Table C1. Percentage of students in District U—46 in Illinois who met the spring of grade 1 reading proficiency 
milestone (level J) at each time point, by race/ethnicity and time point, 2016/17-2017/18 


Kindergarten Grade 1 

Winter Spring Winter Spring 
Race/ethnicity and statistic (n = 2,396) (n = 2,420) (n = 2,586) (n = 2,582) 
Asian students 
Number of students who took the assessment 178 184 204 194 
Number of students who met the proficiency milestone 5 23 85 141 
Percent of students who met the proficiency milestone 2.81 12.50 41.67 72.68 
Black students 
Number of students who took the assessment 115 117 141 147 
Number of students who met the proficiency milestone 0 5 19 55 
Percent of students who met the proficiency milestone 0.00 4.27 13.48 37.41 
Hispanic students 
Number of students who took the assessment 1,391 1,404 1,484 1,484 
Number of students who met the proficiency milestone 1 18 196 815 
Percent of students who met the proficiency milestone 0.07 1.28 13.21 54.92 
White students 
Number of students who took the assessment 626 627 670 666 
Number of students who met the proficiency milestone 8 50 181 408 
Percent of students who met the proficiency milestone 1.28 7.97 27.01 61.26 
Multiracial students 
Number of students who took the assessment 75 76 76 80 
Number of students who met the proficiency milestone 1 2 17 45 
Percent of students who met the proficiency milestone 1.33 2.63 22.37 56.25 
Other race/ethnicity students 
Number of students who took the assessment 11 12 11 11 
Number of students who met the proficiency milestone 1 1 2 4 
Percent of students who met the proficiency milestone 9.09 8.33 18.18 36.36 


Note: Students could meet the milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark Assessment System. The samples 
consisted of all students who took a reading assessment at any time point. 
Source: Authors’ analysis of assessment and student records data for 2016/17—2017/18 from District U-46. 


Students in District U-46 took the English, Spanish, or both versions of the Fountas and Pinnell 
Benchmark Assessment System 

The study team calculated the number of students in District U-46 who took only the English version, only the 
Spanish version, or both versions of the Fountas and Pinnell Benchmark Assessment System (table C2). The 
number of students who took only the English version increased across winter of kindergarten and spring of grade 
1, as did the number of students who took both versions. The number of students who took only the Spanish 
version declined over the same period. Of the 1,088 students who took only the Spanish version in winter of 
kindergarten, a majority took the English version by spring of grade 1 (929 students took both versions and 25 
students took only the English version, for a total of 954 students who completed the English version; figure C1). 
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Table C2. Number of students in District U—46 in Illinois who took only the English version of the Fountas and 
Pinnell Benchmark Assessment System, only the Spanish version, or both, at each time point, 2016/17- 
2017/18 


Assessment version Winter Sola ays4 Winter Spring 
Spanish only 1,088 1,097 985 138 


Note: The samples consist of any students who took a reading assessment at any time point. 
Source: Authors’ analysis of assessment and student records data for 2016/17—2017/18 from District U-46. 


Figure C1. Number of students in District U—46 in Illinois who took only the Spanish version of the Fountas and 
Pinnell Benchmark Assessment System, only the English version, or both versions from spring of kindergarten 
to spring of grade 1 after taking only the Spanish version in winter of kindergarten, 2016/17-2017/18 


Winter (Kindergarten) Students who took only the Spanish version = 1,088 


Spring (Kindergarten) 


Winter (Grade 1) 


Spanish and 


Spring (Grade 1) English = 929 


Source: Authors’ analysis of assessment and student records data for 2016/17—2017/18 from District U-46. 


A smaller percentage of Hispanic students in District U-46 who were English learner students than of 
Hispanic students who were not English learner students met the spring of grade 1 reading 
proficiency milestone 

Because District U-46 had a high percentage of Hispanic students (57 percent)—not all of whom were English 
learner students—the study team conducted additional analyses to determine whether the percentage of 
Hispanic students who met the spring of grade 1 reading proficiency milestone (level J) by the end of grade 1 
varied by English learner status. A smaller percentage of Hispanic students who were English learner students 
(54 percent) than of Hispanic students who were not English learner students (59 percent) met the reading 
proficiency milestone by the end of grade 1 (table C3). 


Table C3. Number and percentage of Hispanic students who met the spring of grade 1 reading proficiency 
milestone (level J) in District U-46 in Illinois, by English learner status and time point, 2016/17-2017/18 


Kindergarten Grade 1 
English learner status and statistic Winter Spring Winter 
Hispanic English learner student 
Number of students who took the assessment 1,099 1,113 1,175 1,175 
Number of students who met the proficiency milestone 0 7 129 632 
Percent of students who met the proficiency milestone 0.00 0.63 10.98 53.79 
Hispanic non-English learner student 
Number of students who took the assessment 292 291 309 309 
Number of students who met the proficiency milestone 1 11 67 183 
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Kindergarten Grade 1 


English learner status and statistic Winter Spring Winter Spring 
Percent of students who met the proficiency milestone 0.34 3.78 21.68 59.22 


Note: Students could meet the proficiency milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark Assessment System. 
The sample consisted of students who took a reading assessment at any time point. 
Source: Authors’ analysis of assessment and student records data for 2016/17—2017/18 from District U-46. 


The percentage of students in District U-46 who were eligible for the national school lunch program 
who met the reading proficiency milestone varied by race/ethnicity 

The study team compared the results for students who were eligible for the national school lunch program and 
those who were not eligible for the program by racial/ethnic group. National school lunch program eligibility is an 
indicator of socioeconomic disadvantage. A higher percentage of Black and Hispanic students were eligible for the 
national school lunch program than of Asian and White students (table C4). The study team performed these 
supplementary analyses to determine whether the results varied by student race/ethnicity when students with 
similar socioeconomic disadvantage were compared and thus whether racial/ethnic disparities in reading 
proficiency existed even after socioeconomic disadvantage was accounted for. 


Table C4. Demographic characteristics of students eligible for the national school lunch program in 
District U—46 in Illinois, 2016/17-2017/18 


Total students Students eligible for the national school lunch program 
Student racial/ethnic group Number Number Percent 
Asian students 217 60 27.65 
Black students 153 129 84.31 
Hispanic students 1,540 1,283 83.31 
White students 698 215 30.80 
Multiracial students 83 32 38.55 
Other race/ethnicity students? 12 6 50.00 
Total number of students 2,703 1,725 63.82 


a. Students who identified as a race/ethnicity that made up less than 2 percent of the population were combined into the other race category. 

Note: Percentages represent students by racial/ethnic group as a proportion of the district’s total number of students in the sample across the two years. 
The sample consisted of students who took a reading assessment at any time point. 

Source: Authors’ analysis of student records data for 2016/17—2017/18 from District U-46. 


Among students eligible for the national school lunch program, Black students were less likely than students of 
the other racial/ethnic groups to meet the spring of grade 1 reading proficiency milestone (level J) by the end of 
grade 1: 35 percent of Black students met the milestone compared with 54 percent of Hispanic students and 
49 percent of White students (table C5). 
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Table C5. Percentage of students in District U—46 in Illinois eligible for the national school lunch program who 
met the spring of grade 1 reading proficiency milestone (level J) at each time point, by race/ethnicity, 
2016/17-2017/18 


Kindergarten Grade 1 
Student racial/ethnic group and statistic Winter Syolaiars4 Winter Syolaiays4 
Asian students 
Number of students who took the assessment 49 52 56 53 
Number of students who met the proficiency milestone 0 0 18 36 
Percent of students who met the proficiency milestone 0.00 0.00 32.14 67.92 
Black students 
Number of students who took the assessment 97 99 120 125 
Number of students who met the proficiency milestone 0 3 12 44 
Percent of students who met the proficiency milestone 0.00 3.03 10.01 35.20 
Hispanic students 
Number of students who took the assessment 1,159 1,171 1,240 1,238 
Number of students who met the proficiency milestone 0 12 152 673 
Percent of students who met the proficiency milestone 0.00 1.02 12.29 54.36 
White students 
Number of students who took the assessment 190 190 198 199 
Number of students who met the proficiency milestone 0 5 38 97 
Percent of students who met the proficiency milestone 0.00 2.63 19.19 48.74 
Multiracial students 
Number of students who took the assessment 25 26 29 32 
Number of students who met the proficiency milestone 0 0 2 13 
Percent of students who met the proficiency milestone 0.00 0.00 6.90 40.63 


Other race/ethnicity students 

Number of students who took the assessment 5 6 6 6 
Number of students who met the proficiency milestone 

Percent of students who met the proficiency milestone . s : a 


Note: Students could meet the proficiency milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark Assessment System. 
The sample consisted of students who took a reading assessment at any time point. 

a. The data in these cells have been suppressed because there were fewer than 10 students in each cell. 

Source: Authors’ analysis of assessment and student records data from District U-46 (2016/17—2017/18). 


In District 186 student demographic groups varied in their average Rasch unit score on the Measures 
of Academic Progress for Primary Grades 

Among students in District 186 who were eligible for the national school lunch program, White students had 
higher average Rasch unit scores on the Measures of Academic Progress for Primary Grades throughout the study 
period than Black students did (figure C2). The difference between the average scores of White students eligible 
for the national school lunch program and Black students eligible for the program in spring of grade 1 was 0.49 
standard deviation units (equivalent to 88 days of instruction). The sample sizes for some demographic groups at 
some time points were very small, so the findings should be interpreted with caution. 
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Figure C2. Among students in District 186 in Illinois who were eligible for the national school lunch program, 
White and Hispanic students had slightly higher average scores than did Black students, 2017/18-2018/19 
Average Rasch unit score on the Measures of Academic Progress for Primary Grades 


<—=@ + Black students eligible for national school lunch program 
=<=@ + Hispanic students eligible for national school lunch program 


tr White students eligible for national school lunch program 
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Note: The sample included kindergarten students in the 2017/18 school year and grade 1 students in the 2018/19 school year in 23 schools who were eligible 
for a reading assessment at any time point and were eligible for the national school lunch program: 184 Black students and 131 White students in fall of 
kindergarten; 366 Black students, 17 Hispanic students, and 215 White students in winter of kindergarten; 352 Black students, 20 Hispanic students, and 
209 White students in spring of kindergarten; 311 Black students, 13 Hispanic students, and 153 White students in fall of grade 1; 351 Black students, 26 
Hispanic students, and 183 White students in winter of grade 1; 354 Black students, 24 Hispanic students, and 189 White students in spring of grade 1. 
Findings for Asian students at all time points and for Hispanic students in fall of kindergarten are suppressed because there were fewer than 10 students at 
each time point. 

Source: Authors’ analysis of assessment and student records data for 2017/18—2018/19 from District 186. 
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Appendix D. Other analyses 

This appendix compares results between the full sample (students who had assessment scores at any time point) 
and the stable sample, the subset of students in District U-46 (Elgin Area Schools) and District 186 (Springfield 
Public Schools) who had assessment scores at all time points. By performing sensitivity analyses with analytic 
samples that remain stable across time, the study team was able to determine whether increases in reading 
achievement for a cohort reflected actual changes in student reading achievement rather than changes in the 
cohort composition resulting from student mobility. The findings for the samples in both districts were similar, 
indicating that results were not due to changes in the composition of the cohorts across time. 


The stable sample for each district was similar to the full sample, which included all students who had assessment 
scores at any time point. In District U-46 there were 2,703 students in the full sample and 2,261 students in the 
stable sample. In District 186 there were 928 students in the full sample and 731 students in the stable sample. 


In District U-46 differences between the full sample and stable sample were small 

The percentages of students in the full sample who met the spring of grade 1 reading proficiency milestone (level J) 
were similar to the percentages of students in the stable sample who met the milestone, both overall and for each 
subgroup analysis. The largest difference between the two samples was in spring of grade 1, when 57 percent of 
students in the full sample met the spring of grade 1 proficiency milestone compared with 60 percent of students 
in the stable sample (table D1). 


Table D1. Percentage of students in District U-46 in Illinois who met the spring of grade 1 reading proficiency 
milestone (level J), by full sample and stable sample and time point, 2016/17—2017/18 


Kindergarten Grade 1 
Statistics and sample Winter Spring Winter Syolaiatsd 
Number of students who took the assessment 
Full sample 2,396 2,420 2,586 2,582 
Stable sample 2,261 2,261 2,261 2,261 
Number of students who met the proficiency milestone 
Full sample 16 99 500 1,468 
Stable sample 16 95 448 1,333 
Percentage of students who met the proficiency milestone 
Full sample 0.67 4.09 19.33 56.86 
Stable sample 0.71 4.20 19.81 59.96 


Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. Students could meet the proficiency milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark 
Assessment System. 

Source: Authors’ analysis of assessment and student records data for 2016/17—2017/18 from District U-46. 


The percentages of students who met the proficiency milestone by student demographic groups was similar for 
the full and stable samples. For English learner and non-English learner students in District U-46, the largest 
difference between the two samples was in spring of grade 1 for English learner students, when 53 percent of 
English learner students in the full sample met the proficiency milestone compared with 56 percent of English 
learner students in the stable sample (table D2). 
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Table D2. Number and percentage of English learner students and of non-English learner students in District 
U—46 in Illinois who met the spring of grade 1 reading proficiency milestone (level J), by full sample and stable 
sample and time point, 2016/17—2017/18 

Kindergarten Grade 1 


English learner student status, statistic, and sample Winter Soldat Winter Spring 
English learner students 


Number of students who took the assessment 


Full sample 1,255 1,277 1,360 1,345 
Stable sample 1,195 1,195 1,195 1,195 
Number of students who met the proficiency milestone 

Full sample 0 9 158 715 
Stable sample 0 8 141 667 
Percent of students who met the proficiency milestone 

Full sample 0.00 0.70 11.62 53.16 
Stable sample 0.00 0.67 11.80 55.82 


Non-English learner students 
Number of students who took the assessment 


Full sample 1,141 1,143 1,226 1,237 
Stable sample 1,066 1,066 1,066 1,066 
Number of students who met the proficiency milestone 

Full sample 16 90 342 753 
Stable sample 16 87 307 666 
Percent of students who met the proficiency milestone 

Full sample 1.40 7.87 27.90 60.87 
Stable sample 1.50 8.16 28.80 62.48 


Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. Students could meet the proficiency milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark 
Assessment System. 

Source: Authors’ analysis of assessment and student records data for 2016/17—2017/18 from District U-46. 


For students in special education and students not in special education in District U-46, the largest difference 
between the two samples was in spring of grade 1 for students who were not in special education. At that time 
point 60 percent of students who were not in special education in the full sample met the proficiency milestone 
compared with 63 percent of students who were not in special education in the stable sample (table D3). 
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Table D3. Number and percentage of students in special education and not in special education in District U- 
46 in Illinois who met the spring of grade 1 reading proficiency milestone (level J), by full sample and stable 
sample and time point, 2016/17-2017/18 

Kindergarten Grade 1 


Special education status, statistic, and sample Winter Spring Winter Spring 
Students in special education 
Number of students who took the assessment 


Full sample 268 274 272 270 
Stable sample 243 243 243 243 
Number of students who met the proficiency milestone 

Full sample 1 6 22 75 
Stable sample 1 6 20 71 
Percent of students who met the proficiency milestone 

Full sample 0.37 2.19 8.09 27.78 
Stable sample 0.41 2.47 8.23 29.22 


Students not in special education 
Number of students who took the assessment 


Full sample 2,128 2,146 2,314 2,312 
Stable sample 2,018 2,018 2,018 2,018 
Number of students who met the proficiency milestone 

Full sample 15 93 478 1,393 
Stable sample 15 89 428 1,262 
Percent of students who met the proficiency milestone 

Full sample 0.70 4.33 20.67 60.25 
Stable sample 0.74 4.41 21.21 62.54 


Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. Students could meet the proficiency milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark 
Assessment System. 

Source: Authors’ analysis using assessment and student records data for 2016/17—2017/18 from District U-46. 


The percentages of students who met the proficiency milestone were also similar in the full and stable samples in 
District U-46 for students who were eligible for the national school lunch program and for students who were not 
eligible. The largest difference between the two samples was in spring of grade 1. At that time point 52 percent 
of students who were eligible for the national school lunch program in the full sample met the proficiency 
milestone compared with 55 percent of students who were eligible for the national school lunch program in the 
stable sample (table D4). 
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Table D4. Number and percentage of students in District U—46 in Illinois eligible for the national school lunch 
program and not eligible for the program who met the spring of grade 1 reading proficiency milestone 
(level J), by full sample and stable sample and time point, 2016/17-2017/18 

Kindergarten Grade 1 


National school lunch program status, statistic, and sample Winter Spring Winter 
Students eligible for the national school lunch program 


Number of students who took the assessment 


Full sample 1,525 1,524 1,426 1,653 
Stable sample 1,439 1,439 1,439 1,439 
Number of students who met the proficiency milestone 

Full sample 0 20 223 866 
Stable sample 0 18 203 789 
Percent of students who met the proficiency milestone 

Full sample 0.00 1.30 13.52 52.39 
Stable sample 0.00 1.25 14.11 54.83 


Students not eligible for the national school lunch program 
Number of students who took the assessment 


Full sample 871 876 937 929 
Stable sample 806 822 822 822 
Number of students who met the proficiency milestone 

Full sample 16 79 277 602 
Stable sample 16 77 245 544 
Percent of students who met the proficiency milestone 

Full sample 1.84 9.02 29.56 64.80 
Stable sample 1.95 9.37 29.81 66.18 


Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. Students could meet the proficiency milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark 
Assessment System. 

Source: Authors’ analysis using assessment and student records data for 2016/17—2017/18 from District U-46. 


The percentages of students who met the proficiency milestone by racial/ethnic group in District U-46 were also 
similar in the full and stable samples. The largest difference in the percentage of students meeting the reading 
proficiency milestone between the two samples was for Asian students in winter of grade 1, when 42 percent of 
Asian students in the full sample met the proficiency milestone compared with 44 percent of Asian students in 
the stable sample (table D5 and figure D1). 
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Table D5. Percentage of students in District U-46 in Illinois who met the spring of grade 1 reading proficiency 
milestone (level J), by student race/ethnicity, full sample and stable sample, and time point 2016/17-2017/18 
Kindergarten Grade 1 


Student race/ethnicity, statistic, and sample Winter Syolaiars4 Winter Spring 


Asian students 
Number of students who took the assessment 


Full sample 178 184 204 194 
Stable sample 160 160 160 160 
Number of students who met the proficiency milestone 

Full sample 5 23 85 141 
Stable sample 5 22 71 120 
Percent of students who met the proficiency milestone 

Full sample 2.78 12.50 41.67 72.68 
Stable sample 3.13 13.75 44.38 75.00 


Black students 
Number of students who took the assessment 


Full sample 115 117 141 147 
Stable sample 106 106 106 106 
Number of students who met the proficiency milestone 

Full sample 0 5 19 55 
Stable sample 0 5 13 40 
Percent of students who met the proficiency milestone 

Full sample 0.00 4.27 13.48 37.41 
Stable sample 0.00 4.72 12.26 37.74 


Hispanic students 
Number of students who took the assessment 


Full sample 1,391 1,404 1,484 1,484 
Stable sample 1,330 1,330 1,330 1,330 
Number of students who met the proficiency milestone 

Full sample 1 18 196 815 
Stable sample 1 17 182 759 
Percent of students who met the proficiency milestone 

Full sample 0.07 1.28 13.21 54.92 
Stable sample 0.08 1.28 13.68 57.07 
White students 

Number of students who took the assessment 

Full sample 626 627 670 666 
Stable sample 587 587 587 587 
Number of students who met the proficiency milestone 

Full sample 8 50 181 408 
Stable sample 8 48 164 371 
Percent of students who met the proficiency milestone 

Full sample 1.28 7.97 27.01 61.26 
Stable sample 1.36 8.18 27.94 63.20 
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Kindergarten Grade 1 


Student race/ethnicity, statistic, and sample Winter Spring Winter 


Multiracial students 
Number of students who took the assessment 


Full sample 75 76 76 80 
Stable sample 68 68 68 68 
Number of students who met the proficiency milestone 

Full sample 1 2 17 45 
Stable sample 1 2 16 39 
Percent of students who met the proficiency milestone 

Full sample 1.33 2.63 22.37 56.25 
Stable sample 1.47 2.94 23.53 57.35 


Other race/ethnicity students 
Number of students who took the assessment 


Full sample 11 12 10 11 
Stable sample 5 5 5 5 
Number of students who met the proficiency milestone 

Full sample 1 1 1 2 
Stable sample 7 @ a a 
Percent of students who met the proficiency milestone 

Full sample 9.09 8.33 10.00 18.18 
Stable sample a 2 a 7 


Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. Students could meet the proficiency milestone in either the English or the Spanish version of the Fountas and Pinnell Benchmark 
Assessment System. 

a. The data in these cells have been suppressed because there were fewer than 10 students in each cell. 

Source: Authors’ analysis using assessment and student records data for 2016/17—2017/18 from District U-46. 


Figure D1. The percentages of students in each racial/ethnic group in District U—46 in Illinois who met the 
spring of grade 1 reading proficiency milestone (level J) at each time point were similar for the full sample and 
the stable sample, 2016/17-2017/18 

Full sample Stable sample 
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Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. Students could reach the proficiency milestone in either the English or Spanish version of the Fountas and Pinnell Benchmark Assessment 
System. The full sample consisted of 2,703 students in 40 schools. In winter of kindergarten the sample included 178 Asian students, 115 Black students, 
1,391 Hispanic students, and 626 White students. In spring of kindergarten, the sample included 184 Asian students, 117 Black students, 1,404 Hispanic 
students, and 627 White students. In winter of grade 1 the sample included 204 Asian students, 141 Black students, 1,484 Hispanic students, and 670 White 
students. In spring of grade 1 the sample included 194 Asian students, 147 Black students, 1,484 Hispanic students, and 666 White students. 

The stable sample consisted of 160 Asian students, 106 Black students, 1,330 Hispanic students, and 587 White students. 

Source: Authors’ analysis using assessment and student records data for 2016/17—2017/18 from District U-46. 


In District 186 differences between the full sample and the stable sample were also small 


Students in District 186 in the full sample and in the stable sample had nearly identical average Rasch unit scores 
on the Measures of Academic Progress for Primary Grades at each time point. The largest difference between the 


two samples was in winter of kindergarten, when the average scores were 149 for the full sample and 151 for the 
stable sample (table D6). 


Scores by student demographic groups were also similar for the stable sample and the full sample. 


For students in special education and students not in special education, average Rasch unit scores were similar 
for the two samples at each time point (table D7). 
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Table D6. Descriptive statistics for students in District 186 in Illinois for average Rasch unit score on the 
Measures of Academic Progress for Primary Grades, by full sample and stable sample and by time point, 
2017/18-2018/19 

Kindergarten Grade 1 


Statistic and sample Winter Spring Winter Spring 
Average Rasch unit score 

Full sample 140 149 157 160 169 174 
Stable sample 141 151 158 161 169 174 
Standard deviation of Rasch unit score 


Fullsample 0 
Stable sample 10 12 14 14 15 16 
‘Number ofstudentswhotookthe assessment 
Stable sample 731 731 731 731 731 731 


Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. 
Source: Authors’ analysis using assessment and student records data for 2017/18-2018/19 from District 186. 


Table D7. Descriptive statistics for students’ average Rasch unit score on the Measures of Academic Progress 
for Primary Grades in District 186 in Illinois, by students’ special education status and by full sample and 

stable sample and by time point, 2017/18-2018/19 
Kindergarten Grade 1 


Special education status, statistic, 


lavemcy-}anyel(=1 Winter Spring Winter Sela ias4 


Students in special education 


Average Rasch unit score 
Full sample 138 146 153 155 161 165 


Stablesample 238 147. 154 156 162 167 
Standard deviation of Rasch unit score 

fulsample 0. dt 4 45 06 
Stablesampless—<‘“‘is‘s*s*s*C~tsSOSCti‘“‘C(‘#®SCSCSC‘C‘((‘SCCSCSCOBB[UOC‘CQNS(C(C#SAACO;*;‘CC™‘CCSUUUOUCMTO” 
Number of students who took the assessment 

fullsample s—i<‘<‘é;!!””~C~é‘ilO!! lU8BS”*~*~«~dBASC(‘<‘OSCC#CAQMHOOO ODOR 
Stablesample ss sti(‘élfOO!!!UlUS!|lU! ll 


Students not in special education 


Average Raschunit score 
Full sample 140 149 157 159 168 174 


Stablesample s—<“‘éS!!t(dMO!! US lB lOSSC*~«<iSSC( SC 
Standard deviation of Rasch unit score 

fulsample si‘; 3OWtlddllUhl 
Number of students who took the assessment 

fullsample i‘ sé;t;!!!!Ct(«!!!! lU8T™~™~COC~<“‘“‘iS:*‘“‘(‘iS:C*‘«CwSSCS 
Stablesample ss ssi<‘;S!ét oO!!! 8B) UIC (sCBC(‘é‘ SCS 
Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 


at all time points. 
Source: Authors’ analysis using assessment and student records data for 2017/18-2018/19 from District 186. 


For students eligible for the national school lunch program and students not eligible, the largest difference 
between the two samples was 1 point, with the full sample lower. For example, students eligible for the national 
school lunch program in fall of kindergarten had an average Rasch unit score of 138 in the full sample and 139 in 
the stable sample (table D8). 
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Table D8. Descriptive statistics for students’ average Rasch unit score on the Measures of Academic Progress 
for Primary Grades in District 186 in Illinois, by eligibility for the national school lunch program and by full 
sample and stable sample and by time point, 2017/18-2018/19 


National school lunch program eligibility, statistic, Kindergarten Grade 1 


lavemcy-]any el(=1 Fall Winter Syolaiatsd Winter Sela ias4 


Students eligible for the national school lunch program 
Average Rasch unit score 


fullsampless—<CSsSsSSSSSSC<CSsti‘CstBSC(ti‘iaA SST STO 
Stable sample 139 148 155 157 165 170 
Standard deviation ofRaschunitscore 
Full sample 8 11 12 12 14 15 
stable sample __E__ _ 
NN 
Full sample 393 722 703 568 674 678 
Stable sample 314 538 539 469 532 533 


Students not eligible for the national school lunch program 


Average Rasch unit score 


fullsamplesi(i‘ié;!!!!!!!!!! (CO 8K 2 CCiaC( tid 
Stable sample 141 155 163 163 173 179 
Standard deviation ofRaschunitscore 
Full sample 10 12 15 14 16 18 
Stable sample 10 11 16 13 15 16 
Number of studentswhotooktheassessment 
Full sample 411 155 155 242 149 150 
Stable sample 316 129 130 178 122 125 


Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 
at all time points. 
Source: Authors’ analysis using assessment and student records data for 2017/18-2018/19 from District 186. 


For the racial/ethnic groups most average scores were the same or differed by just 1-2 points between the two 
samples (table D9). The largest difference was for Asian students in fall of grade 1, when average Rasch unit scores 
were 175 in the full sample and 170 in the stable sample. 


Table D9. Descriptive statistics for students’ average Rasch unit score on the Measures of Academic Progress 
for Primary Grades in District 186 in Illinois, by racial/ethnic group, full sample and stable sample, and time 
point, 2017/18-2018/19 


Student race/ethnicity, statistic, 


Kindergarten Grade 1 


elavemcy-}an)e)(=1 Winter Spring Winter 


Asian students 
Average Rasch unit score 


fulsample 147 162 163 4.17 180 188 
Stablesample #148 1165 167 «+170 &42+176 °° ©185 
Standard deviation of Rasch unit score 

hiante  .»x+« so ££. 4a a a 
Stablesample 4 4 nt Wm Bb #W. 
‘Number of studentswhotookthe 
assessment 

fullsample 44 3 15 2 2 23 
Stablesamle. 2 #2 |. « tt 2 eae a 
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Student race/ethnicity, statistic, Kindergarten Grade 1 


elavemcy-}an)el(=) Winter Syolaiays4 Winter 
Black students 


Average Rasch unit score 
Full sample 137 145 153 156 164 168 


Stablesample 138 14 154 156 164 168 
Full sample 9 11 13 13 14 14 


Number of students who took the 
assessment 
Full sample 401 413 400 406 403 404 


Stablesample 345—(“(‘ WS 8H 8H 8H 8S 


Hispanic students 


Average Raschunit score 
Full sample 142 150 158 161 168 173 


Stablesample —i(‘él! !!!lUS)|6hlO!)!™©6hltC“<ié‘iSSSC(;SCS 
Standard deviation ofRaschunitscore 
Full sample 10 11 14 16 18 19 
Stable sample 11 11 15 15 18 18 


Number of students who took the 
assessment 
Full sample 29 29 32 33 36 35 


Stablesample #24. 2 2m 2 420. 2. 
White students 


Average Rasch unit score 
Full sample 143 153 162 165 173 179 


Stablesample 144 155 163 16 174 180 
Full sample 9 12 14 14 16 17 


Number of students who took the 
assessment 
Full sample 341 351 341 333 329 332 


Stablesample 269. 269 269 £267 «1267 «267 
Multiracial students 


Average Rasch unit score 
Full sample 139 150 157 161 171 176 


Stablesample 440 152 159 162 172 I. 
Standard deviation ofRaschunitscore 
Full sample 10 11 13 15 15 16 
Stable sample 10 10 12 14 14 15 


Number of students who took the 
assessment 
Full sample 140 139 137 131 134 131 


Stable sample 110 110 110 105 106 106 
Note: The full sample includes students who had assessment scores at any time point, and the stable sample includes the subset of students who had scores 


at all time points. 
Source: Authors’ analysis using assessment and student records data for 2017/18-2018/19 from District 186. 
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